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We study adaptive dynamics in games where players abandon the population at a given rate, 
and are replaced by naive players characterized by a prior distribution over the admitted strategies. 
We demonstrate how such process leads macroscopically to a variant of the replicator equation, 
with an additional term accounting for player turnover. We study how Nash equilibria and the 
dynamics of the system are modified by this additional term, for prototypical examples such as 
the rock-scissor-paper game and different classes of two-action games played between two distinct 
populations. We conclude by showing how player turnover can account for non-trivial departures 
from Nash equilibria observed in data from lowest unique bid auctions. 
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I. INTRODUCTION 



Perhaps the most important skill for competing agents is the ability to adapt to a changing environment by 
constantly assessing and modifying their behavior. Consequently, while game theory has been initially mostly focused 
on the study of equilibria Q3I2]> the study of adaptive dynamics has acquired more and more relevance in recent years. 
Various models have been put forward to capture the learning processes of competing individuals and the resulting 
evolution of the population |3H5| ■ A key result is that many agent-based algorithms of adaptive dynamics allow for a 
simple macroscopic description in terms of replicator equations [6 - 9 . In the replicator dynamics, a large population of 
individuals participate in a game. Let us call Xi the fraction of population playing a given strategy, where admissible 
strategies are labeled as % = {1, 2, 3, . . .}. The time evolution of such fractions is given by: 

^-Xi = Xi(TTi(x) - 7r(x)), (1) 

at 

where 7Ti(x) is the frequency-dependent payoff of strategy i, and 7f = J2i x i 7r i is t ne average payoff. In this setting, 
the initial condition x° represent a prior distribution of strategy preferences, before the adaptation process takes 
place. Approaches based on Eq. ([!]) have proven successful in describing systems within biology as well as economics 
[SI Uni E] . It can be easily shown that stable equilibria of replicator dynamics correspond to Nash equilibria, where 
no individual can benefit from changing strategy unilaterally jTJ 1 1 2] . 

Usually, in adaptive dynamics, one has in mind a fixed population of players that acquire experience over time, or 
biological populations, where offsprings inherit strategies from their parents. However, one can think of a number of 
concrete examples where games are played in a more open setting, with the possibility of players to leave the game and 
be replaced by less experienced ones. On the market, new companies are founded while old companies collapse. Within 
companies, experienced employees retire so that young graduates may start their career. On general grounds, one 
should expect turnover to have a profound impact on the dynamics of the game and lead to a rich phenomenlogy. In 
a standard adaptive dynamics all individuals within the population have the same degree of experience. Conversely, 
here each agent sees a non-trivial mixture of players with different experience levels. While the Nash equilibrium 
strategy will be optimal against very experienced players, it would not necessarily be the most effective to exploit 
naive newcomers. Therefore, one can expect adaptive dynamics in this case to converge to equilibria being different 
from Nash equilibria, and being crucially affected by the rate of turnover, which in turns determines the steady-state 
structure of the population both in terms of experience and strategies. In such framework, interesting information 
can be obtained by dissecting the experience composition of each strategy, for example to assess which experience 
classes are performing better in the game in terms of payoff. 

In this paper, we demonstrate how taking into account turnover of players leads macroscopically to a novel variant 
of the replicator dynamics. We present a derivation of the replicator equation with turnover (Eq. [6]) in Section [IT] 
In the remainder of the paper, we apply this equation to analyze the effect of turnover in simple evolutionary games. 
We begin with the simple paradigmatic cases of the rock-paper-scissors game and the set of two-action games played 
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between two different populations. In the latter case, we show how increasing the turnover rate can lead to abrupt 
change in the equilibrium state caused by bifurcations in the corresponding dynamical system. We conclude the paper 
by showing how this approach can provide an interpretation for the observed bid distribution in online lowest unique 
bid auctions. 



II. MODEL 

We aim at generalizing the replicator equations ([!]) to situations in which agents are replaced by inexperienced 
individuals at a rate p. Let a large population of players engage in a game with strategies labeled i — {1, 2,3,.. .}. 
We divide the players in experiences classes: let n,-(r,f) be, at time t, the number of players having been in the game 
for a time r and playing strategy i. The normalization condition for the n's reads: 



oo 

n,(r,t)=JV Vt, (2) 

i t=0 

where TV is a fixed total population size. We also define the total fraction of players adopting strategy i at time t, 
x i{t) — S^Lo n »( T i t)/N. As in the standard replicator equation |lj), we introduce the average payoff of strategy i, 
7Tj(x(i)), and the average payoff across strategies tt — J2i x i n i- 

For the sake of simplicity, we consider a simple adaptive dynamics in which individuals learn of alternative strategics 
and their average payoff at a rate equal to the fraction of the population that plays by this strategy. Further, an 
individual playing strategy i learning of a strategy j with a higher average payoff, changes to strategy j with a 
probability proportional to the payoff difference. Combining these assumptions gives an overall rate of change from 
strategy i to strategy j equal to n^Xj^j — 7Tj), if ttj > 7q. 

Furthermore, agents leave the population at rate p and are replaced by inexperienced agents. The new agents play 
each strategy i with a probability proportional to a given distribution x\. 

We wish to study the time development of the ni(r,t). The learning dynamics encoded in the rules above reads: 



n;(r + l,t + l)i - nj(r,t) = (1 - p) \ ^ nj(Y, t)x l (t)(-K l - tt,) - ^ n^r, t)xj(t)(-Kj - iTi) J - pm(T,t), (3) 

yj:7Tj<7r t j:-K j>TTi J 

where the first sum represents players changing to strategy i from strategies with lower payoffs, and the second sum 
represents players changing from strategy i to strategies with higher payoffs. Performing a continuous time limit, 
the left hand side of the above equation becomes dtn,i(T,t) + d T ni(r,t). We can now integrate the continuous time 
version ofEq. (§ over r and divide by the population size N, to obtain a closed evolution equation for the strategies 
Xi(t). Let us recall that rii(0,t) = aNx®, where the proportionality constant a can be determined by imposing that 
the final equation preserves the normalization condition. We also assume that, due to the effect of turnover, one has 
lim T ->oo ni(r,i) — Vi,t. After combining the two sums, this results in 

^■^(t) = (1 - p)y^ j Xj(t)x i (t)(7r i - TTj) + ax\ -pxi. (4) 

3 

Imposing the normalization results in a = p. This also implies that the density of players having a given experience 
level at steady state is exponentially distributed, oo) = Npexp(—pr). If we furthermore use the normalization 

condition Y], Xj = 1 and the definition of the average payoff tt, we get 

— Xi = (l-p)x i {-K i - 7f) +p(x° i - Xi). (5) 

Upon rescaling time by 1 — p and define a rescaled turnover rate x = p/0- — P), wc finally obtain 

^-Xi = Xifa - 7f) + X(X° - Xi). (6) 

Eq. ([6]) constitutes the starting point of our analysis. It should be clear that our specific choice of adaptive dynamics 
is not crucial for deriving Eq. (J6|, and that the same macroscopic limit could be obtained for other microscopic 
adaptation rules leading to the replicator equation ([!]) in the absence of turnover. Eq. Q can be of course also 
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derived (or justified) heuristically, for example by making an analogy with a (damped) driven dynamical system, 
where the prior distribution x® in the right hand side acts as a forcing term. Note that Eq. ^ can be formally recast 
in replicator form: 

^Xi = X i {-K l - 7f) + X(X° ~ Xi) = Xi{ni - 7f), (7) 

where the "effective payoff" of strategy i, %i, is defined as 

Zi = n + x(j^-A- (8) 

In Eq.j7| the average payoff tt does not contain a contribution from the second term in Eq. Q as its average is zero, 
so that 5J~; XtTTi = XiHi. The mapping in Eq. Q is valid only in the interior of the simplex x% > Vi, Xi = I, 
while at the boundary the effective payoffs diverge. This situation has some resemblance to the case of evolutionary 
dynamics in the presence of mutations [7J 113) . The main difference is that in our case the "mutants" are not 
characterized by new, pure strategies or random strategies, but the mixed distribution x® of the existing strategies. 

The fact that the replicator equation with turnover ^ can be rewritten in replicator form Q implies that the two 
equations share several mathematical properties. Among these is that the mean effective payoff for the population can 
not decrease along any trajectory. If the payoff function is continuous and bounded, it therefore serves as a Lyapunov 
function |I4j . which guarantees that all game dynamics either evolve to an equilibrium point or a closed orbit. For the 
replicator equation without turnover, x = 0, all Nash equilibria of a game are equilibrium points of the dynamics [15j . 
For positive %, the equilibrium points are generally different from the Nash equilibria, and we call these the turnover 
equilibria of the game. In the limit x — > oo, agents are insensitive to the rewards and the initial strategy x® is the 
only equilibrium point. 



III. ROCK-PAPER-SCISSORS 



We now apply the concept of agent turnover to the three-strategy game of rock-paper-scissors. In this well-known, 
biologically relevant game [Tfil - EO] . a large population of players can choose between the strategies rock, paper, and 
scissors. The average fraction of the population employing each strategy is denoted xr, xp, and xs, respectively. The 
players are paired randomly in each round. Since the strategies dominate each other cyclically, their payoffs are given 
by 

= x s -x P (9) 
n P = x R - x s (10) 

TT S = X P ~X R . (II) 

Rock-paper-scissors is a zero-sum game, so the average payoff in ^ is always zero. The only Nash equilibrium of the 
game is when all individuals play randomly, giving them all an average payoff of zero |21j . 

When we introduce a player turnover, naive players that enter the game choose to play rock, paper, or scissors with 
probability x° R , x° P: and x^, respectively. Gradually, each individual change strategy according to Eq. (J3|) , resulting 
in the mean behavior given byEq. ©. 

In Fig. [f]^-D the time development of the average strategy played is shown for an initial strategy x° = (0.8, 0.1 , 0.1) 
biased towards rock and for different values of x- I n the absence of agent turnover, the strategy of the population 
oscillates around the Nash equilibrium [211 122j . Upon introducing turnover, the initial overrepresentation of players 
playing rock makes paper a rewarding strategy. As players change strategy towards paper, scissors becomes more 
rewarding, and so on. In the limit of x — > oo, all players remain naive, so the strategy of the population is equal to 
the initial strategy. 

For intermediate turnover rates, the system undergoes damped or critically damped oscillations towards the turnover 
equilibrium, where the gradual learning of the players staying in the game exactly balances the exchange of experienced 
players with naive players. Thus, agent turnover stabilizes the game dynamics, in a similar way as spatial organization 
of agents [TTJ [2H [23] ■ The location of the turnover equilibrium depends non-trivially on the strategy of naive players 
and on the turnover rate, as shown in Fig. [l^. When x is increased from to oo, the turnover equilibrium shifts from 
the Nash equilibrium to the strategy distribution of inexperienced players along a curved line. This means that, for 
a generic value of x, the turnover equilibrium is not trivially a linear combination of the Nash equilibrium and the 
naive strategy. 
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FIG. 1: (Color online) Turnover of players in the rock-paper-scissors game with an initial strategy of 80% rock and 10% paper 
and scissors. A For x — 0, the average strategy oscillates in a fixed distance from the Nash equilibrium. B For infinite 
turnover, only the initial strategy x° is employed. C For intermediate turnovers, the average strategy converges to the turnover 
equilibrium. D The turnover equilibrium for different values of X- The equilibrium is displaced towards paper, which dominates 
the initial strategy. E In the turnover equilibrium, inexperienced players quickly turn away from playing rock. Experienced 
players only play paper. F Inexperienced players have a large negative payoff, which monotonically increases as the players 
gain experience. However, since the number of experienced players falls off exponentially, most games are won by players of 
intermediate experience. In this example, we chose a total population size N = 20. 



In a traditional game of rock-paper-scissors, the time-averaged payoff of all players is zero in any steady state 
situation. In the turnover equilibrium, however, the average payoff of players increases monotonically with their 
experience, such that new, naive players tend to lose to more experienced players. Figure [TjEC shows the average 
strategy nj(r, oo)/N as a function of the experience level r, where the turnover is x = 0-25 and the naive strategy is 
the same as before . The strategy with the highest payoff is paper, which is the only strategy that very experienced 
players employ. However, due to the turnover of agents, the number of experienced players falls off exponentially. 
This causes a radical qualitative difference between the payoff of a given player having an experience level r and the 
total payoff of players in a given experience level. Denoting the turnover equilibrium by x*, the total payoff collected 
by players with experience r is given by ni(r 1 00)71^ (x*), while the payoff of a single player with this experience is 
^2i n i( T : 0O ) 7r i( x *)/(X^i n i( T > oo)).... The two different payoffs are shown in Fig. Jl^ as a function of the experience 
level t for a population of 20 players. The figure demonstrates the counterintuitive fact that, while the payoff of 
each single player increase monotonically with the experience, the majority of the games are won by players with 
intermediate experience levels. 
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IV. TWO-AGENT TWO-ACTION GAMES 

We now turn to the broad class of games, where two populations can both choose between two strategies. Let us 
define the mixed strategies as (x, 1 — x) for the first population and (y, 1 — y) for the second population. The payoffs 
obtained by the two populations depend on their joint actions [15 and are traditionally encoded in payoff matrices 
A and B : 



an °i2 W V 
a 2 i a 22 / \1 - y 

hi b 12 \ ( x 
hi hi) 1 1 - x 



(12) 
(13) 



where nf is the payoff of the first strategy for the first population, and so forth. The two populations are learning 
concurrently and their turnover rates Xx and Xy can i n principle be different. The learning dynamics can be obtained 
from Eq. Q. 

j t x = a:(l - x)(nf - Trf) + Xx(x° - x) (14) 

|y = i/(i-w)W-^)+Xv(w°-i/) (15) 

where the equations for the second strategies can be simply obtained from the normalization conditions. In the 
absence of turnover, this system is known as a bi-matrix replicator equation |151 124) . In the turnover equilibrium, 



which we denote as (x, y) — (x*,y*), the time derivatives are equal to zero. Upon substituting the expressions (12) 



and ( 13 1 for the payoffs leads to the equations 



where we have introduced 



ay* + ni 2 - a 22 = x a _ (16) 

(3x* + bi 2 - b 22 = Xv f ( ~ V [y (17) 

2/ (1 - V ) 



a = an + a 22 — a 21 — a i2 , (18) 
P — hi + b 2 2 - hi - h2- (19) 



The number of solutions to equations (16) and (17) depends on the sign of the product a/3. When aj3 < 0, there 
is always one mixed turnover equilibrium. When a/3 > 0, the number of turnover equilibria can increase at critical 
values of the turnover parameter. The derivation of these result can be found in appendix [A] In the following, we 
provide examples for two well-known two-action games belonging to the two categories aft < and a/3 > 0: the game 
of matching pennies and a coordination game, respectively. 

A. Matching pennies 

A simple example of a/3 < is the zero sum game of matching pennies. Two players secretly turn a penny each to 
heads or tails and reveal the coins simultaneously. If the pennies match, player one receives a reward r from player 
two. If they do not, player one pays the reward to player two. This game is characterized by the payoff matrices 



The learning dynamics ( 14 ) and ( 15 ) with no turnover leads to a marginally stable Nash equilibrium surrounded 
by concentric closed orbits (see Fig. j^). When a positive turnover of players is introduced, the Nash equilibrium is 
perturbed to a stable turnover equilibrium (x*,y*), as displayed in Fig. B]. Inserting the payoff matrices into the 
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FIG. 2: (Color online) Dynamics of the matching pennies game with r — 2 and initial strategy x° = y° — 0.3. A For x — 0, 
the average strategy oscillates at a fixed distance from the Nash equilibrium. B When a turnover is introduced, the average 
strategy converges to the turnover equilibrium. Here Xx = Xv = !• ^ Expected payoff for population one in the turnover 
equilibrium for varying turnovers. Above a critical value of \v — 2, the first population will win most games. 



conditions for turnover equilibrium (16) and (17) gives 



1 Xx x* - x" 

2 4rx*(l-x*)' 

1 * 

1 Xy y -y 

2 4r y*(l-y*)' 



(21) 
(22) 



Matching pennies is a zero sum game. The expected payoff of population one is positive if the two populations 
have a tendency of both playing heads or both playing tails, while it is zero if and only if one of the populations 
chooses their strategy randomly. From Eq. (21) and (22) we see that this happens in the turnover equilibrium if 
(x*,y*) — (1/2, y°) or (x* , y*) = (x° , 1/2). For a given set of initial strategies of the populations, this corresponds to 
the critical turnover rates 



Xy,c — 



r l-2y° 
r i - 2x° 
1 - 2ir n 
1 - 2j/° ' 



(23) 
(24) 



Equation ( 23 1 can only be satisfied if population two wins most games when the initial strategies are employed. In 



this case, the expected payoff of population one is negative if it has a high turnover, since its equilibrium strategy 
is close to its initial strategy. If the turnover is decreased below the critical value (23), the first population starts 



winning more games than it loses regardless of the turnover of population two. Even if population two has an even 
lower turnover, and thus has an initially superior strategy and more time to gain experience, the equilibrium state 
remains advantageous for population one. 
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Likewise, equation (24) only gives a positive critical turnover if population one wins most games when the initial 
strategies are employed. Here, the payoff for population two goes from being positive to negative when its turnover 
is increased past the critical value given by (24), regardless of the turnover of the first population. In Fig. [2p, 
the average payoff for population one is shown for varying turnovers of both populations and the initial strategies 
x° = y° = 0.3. In this case, if the turnover of population two is larger than Xy,c — 2, population one wins most games. 



B. Coordination game 

When the payoff matrices of the two populations equal each other, A = B, one must necessarily have a/3 > 0. One 
example of this is a coordination game, where the two players strive to play the same strategy. We consider the payoff 
matrices 



A = B=(«fy, (25) 

where a = /3 = 5. Without player turnover, this game has two pure Nash equilibria where the populations employ 
the same strategy. In addition, there is a mixed Nash equilibrium at (x*,y*) = (2/5,2/5) with one stable and one 
unstable manifold. The stable manifold constitutes the boundary between the basins of attraction of the two pure 
Nash equilibria (see Fig. [3j\). For sufficiently small turnovers, there are three turnover equilibria close to these three 
Nash equilibria. The initial strategies (x°,y°) determine which equilibrium state the system goes to. 

Let us consider a situation of strong initial disagreement {x , y ) = (0.9,0.1), where inexperienced players of 
population one and two tend to employ the first and second strategy, respectively. We fix Xy —2, while varying \ x as 
a control parameter. In this case, the point (x,y) = (0.4,0.9) is always an equilibrium of the dynamics. At a critical 
value, Xx ~ 0.71, the boundary between the basins of attractions passes the initial strategy (x°,y°) (see Fig. [3^3-C), 
and the steady state changes discontinuously from being dominated by strategy two to strategy one. At the same 
point, the payoff of both populations increases drastically. 

Upon increasing the turnover of the first population even further, the basin of attraction of strategy two suddenly 
disappears at Xx ~ 0.91. For this value, the saddle node equilibrium annihilates with the turnover equilibrium of 
strategy two (see Fig. J3p-E). For larger values of Xx, there is only one turnover equilibrium. In this case the number 
of turnover equilibria changes through a saddle node bifurcation, but for other payoff matrices A, B, a pitchfork 
bifurcation could occur (see appendix |A]) . 

A full bifurcation diagram is shown in Fig. [3J 7 , and the expected payoff of population one is shown in Fig. [3]GI as a 
function of the turnover rates. The abrupt change in payoff at a critical set of turnover rates can clearly be seen. The 
expected payoff of population two follows a similar pattern. In general, a low turnover of either population results in 
higher payoffs of both populations, since a larger proportion of the players has enough experience to mainly bid on 
the dominating strategy. However, increasing the turnover may change which equilibrium the system goes to, leading 
to an increase in the payoff of both players. 



V. LOWEST UNIQUE BID AUCTIONS 



Lowest unique bid auctions have become popular online games, and have recently attracted attention from the 
scientific community [25-27]. In this game, N players pay an entrance fee to independently bid on an item, e.g. a 
car. The winner of the auction is the player with the lowest bid not also plaved by another player. The strategies 
available to each player are the possible integer bids i = {1, 2, 3 . . .}. 

If a population of players place their bids stochastically according to the distribution xt, the average payoff of each 
strategy is, in the large N limit, proportional to [27] 



TT, 



e-N**Y[(l-Nx i e- Nx <). 



(26) 



The Nash equilibrium corresponding to the payoffs of Eq. 26 was compared with data from the auctions website 
auctionair.co.uk in in [27] . It was found an excellent agreement between the Nash equilibrium and the data for low 
to intermediate values of N. In larger auctions, the bidding distribution departed from the Nash equilibrium and was 
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FIG. 3: (Color online) Dynamics of a coordination game with initial strategy (x°,y°) = (0.9,0.1). A Without turnover, the 
game has one saddle point Nash equilibrium with stable manifolds that separate the basins of attraction of two pure Nash 
equilibria. B-C When the initial strategy goes from one basin of attraction to another, the resulting equilibrium state changes 
discontinuously. Here Xy — 2 and \ x — 0.7 and 0.72, respectively. D-E At a critical set of turnover rates two turnover equilibria 
annihilates in a saddle node bifurcation. Here \ y = 2 and \x = 0.91 and 0.92, respectively. F Bifurcation diagram for turnover 
equilibria. The point (a;, y) = (0.4, 0.9) is always an equilibrium. We observe a transcritical bifurcation at \ x ~ 0.4 and the 
saddle node bifurcation from panels D-E at \x ~ 0.9. G Expected payoff of population one in the turnover equilibrium for 
varying turnovers. The dramatic change in payoffs between panel B and C can clearly be seen. 
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FIG. 4: (Color online) Bidding frequencies in online lowest unique bid auctions compared to the theoretical Nash equilibrium 
and the turnover equilibrium for a constant \ = 0.008. Different panels show auctions with items of different value and different 
numbers of players. For small auction sizes both the Nash and the turnover equilibrium fit the empirical bidding distributions. 
In larger auctions the turnover equilibrium fits the data much better than the Nash equilibrium. This is due to a long adaptation 
time for players in large auctions. 



more resemblant of an exponential distribution 

A = £^ r <m 

where the fitted value of the exponential constant was /3 ~ 0.02. On general theoretical grounds, one can argue that 
the exponential bidding strategy is the expected prior distribution for inexperienced players, with limited information 
about the game, that want to avoid the cost of a large bid [571 HE] ■ 



Using (26 1 and (27), we test here the hypothesis that the apparent change of behavior from small to large auction 
sizes is caused by agent turnover. Note that changing the turnover rate x m f6| is equivalent to multiplying the payoff 
function p6| with a constant and rescaling time. As it is also problematic to infer a natural time scale of adaptation 
from the data, we leave the value of x as a fitting parameter. 

Figure [4] compares empirical data from 50 online auctions with varying number of players to the respective Nash 
equilibria and turnover equilibria for a fitted value of x = 0.008. In small auctions, both the Nash equilibrium and 
the turnover equilibrium fit the data well, but the latter also captures the fat tail of the empirical distribution. In 
larger auctions, the Nash equilibrium fails to predict the observed distribution of bids, while the turnover equilibrium 
fits the data remarkably well. 

It should be reminded that, while the exponential strategy distribution for large N is plugged into the model 
empirically as a prior, it is still remarkable that, for the same value of Xi the resulting equilibrium is relatively close 
to the prior for large N and closer to the Nash equilibrium for smaller values of N. The reason stems from the growth 
of the adaptation time of the replicator dynamics with the auction size N (see |27j). The consequence if, even at 
equal turnover rate x, players have sufficient time to adapt for small auction size but not for larger auctions. Clearly, 
one could get an even better fit by letting x vary as a function of N, as one could have different kinds of players and 
therefore different turnovers for different auction types, as exemplified in Appendix[B] We do not present a systematic 
study of the dependence of x on N, as it is beyond the scope of this paper. Our main point here is to show that, 
even in the simple case of a fixed x> the replicator equation with turnover is useful to model a real-life game and can 
help conceptualizing non-trivial discrepancies between observed strategy distributions and Nash equilibria. Finally, 
in Appendix [B] we show that, in the case of larger auctions, the distance from the Nash equilibrium does not seem 
to depend on time. This fact further supports the presence of a steady effect, like the turnover mechanism proposed 
here, preventing the population to reach the Nash equilibrium. 
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VI. DISCUSSION AND CONCLUSION 



In this paper we have extended the theoretical framework of the replicator equation by considering the effect of 
agent turnover, a phenomenon that is in principle present in many game theoretical contexts. Our model provides 
a simple and compact description of the dynamics of the average strategies in the population, and at the same time 
allows for dissecting the behavior of players having different experience levels. The study of experience-dependent 
payoffs can bring to counterintuitive results, as we demonstrated in the simple case of a Rock-Paper-Scissor game. 
When turnover is introduced, the game dynamics become richer. We have shown that it is possible to encounter 
bifurcations of turnover equilibria, and that the equilibrium strategy can change discontinuously with the turnover 
rate. 

The simplicity and flexibility of the approach proposed in this paper can be of use to analyze several games of 
practical importance. For example, we have shown here how turnover of agents can help explaining adaptation in 
online games, where experienced players may exploit the naive strategies of new players. Our comparison with a 
dataset of bidding distributions in online lowest unique bid auctions shows that our model is able to describe the 
empirical steady state distributions far better than the Nash equilibrium. We expect that similar results can be 
obtained by studying data from other games. 

The concepts presented here suggest an interesting analogy with the idea of cognitive hierarchy |29H31j in human 
learning, where players are classified according to the number of reasoning steps they are able to make. In this 
scheme, non-reasoning players are reminiscent of inexperienced players in the present model, and players with more 
reasoning steps can be compared more to experienced players. In this perspective, the model proposed here provides 
a framework in which a cognitive hierarchy emerges naturally from the dynamical equilibrium between adaptation 
and turnover. Indeed, a cognitive hierarchy model has previously been applied to describe the bidding distributions 
lowest unique bid auctions |26j . In the authors' of Ref. [2 6) own words, "the cognitive hierarchy model . . . should be 
viewed as a potential stepping stone to an investigation using a formal learning model". Our work is one proposal of 
such a learning model. 

It is known that a fraction of non-rational, influenceable players may change the game dynamics considerably [321 - 
153] . In a recent study [35], a population of fish were made up of three subgroups: a large group of fish with a small 
preference for going to one place in the aquarium, a smaller group with a strong preference for another place, and a 
group of untrained fish with no prior preference. The study showed, both through simulations and experiment, that 
there exists a critical size of the untrained group above which the entire population goes to the place prefered by 
group one, and below which the minority of group two dictates where the population goes. This resembles our results 
for two-agent two-action coordination games, where a critical turnover of agents changes the turnover equilibrium 
from being dominated by one strategy to the other. 

In our model we assume a constant turnover of agents in time, which leaves one free fitting parameter. However, 
changing the turnover is mathematically equivalent to scaling the expected payoff of all strategies together with a 
time scaling. Since the absolute value of the payoff is often treated as a free parameter when analyzing experiments of 
learning dynamics, this does not increase in general the number of fitting parameters in such an analysis. Furthermore, 
if the absolute values of the payoffs are known, one can in principle measure the turnover rate of agents and avoid 
having a free fitting parameter. 

In conclusion, we have shown how a steady turnover of agents, which is present in most real-life games, can change 
the qualitative dynamics of a game. The simplicity and generality of the framework presented here makes it a natural 
candidate to describe adaptation in a population participating in a game and subject to turnover. 
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Appendix A: Number of turnover equilibria in two-agent two-action games 



In this appendix, we show that the number of possible turnover equilibria in two-agent two-action games is dependent 
on the sign of the product a/3, where a and (3 are given by (18 1 and (19). 

The conditions for turnover equilibrium are given by (16) and (17). Expressing y* as a function of x* , this can be 
written as 
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FIG. 5: (Color online) The number of turnover equilibria in a two-action two-agent game as determined by a graphical solution 
of equation [A2j A When a/3 < the two functions have opposite slopes and hence intersect exactly once in the relevant interval 
< y* < 1. Here the matching pennies game is illustrated with the parameters of Fig. [5)3. B-C When a/3 > the slope of 
the functions have the same sign, and new turnover equilibria can appear through bifurcations. Here the bifurcation in the 
coordination game of Fig. I5J3-C is shown, except that \x is changed from to 0.8 to 1 between panels B-C. 
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where we defined the composite function g{x*). Notice that lim a; *_ > .o 9( x *) = lim^*— g{x*) 
two singularities for values of x* such that y* = and y* = 1. Since < y* < 1, we will focus on the interval of g(x*) 
between the two singularities. 
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Let us investigate the derivative dg/dx*. From Eq. (All and (A2) we get 
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Since x° and y° are probabilities, we always have a; > x° and y° > y° . The fractions in Eq. (A3) and (A4) are 
therefore positive, so the derivative dg/dx* must have the same sign as a. This means that g{x*) is either increasing 
or decreasing monotonically, depending on the sign of a. 
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FIG. 6: (Color online) Same data as Fig. |4j but this time the turnover parameter \ is fitted independently in each plot. The 
fitted value of \ is indicated in each legend. 



If aj3 < 0, the line (3x* + & 12 — 6 2 2 and the function g(x*) have opposite slopes, and therefore intersect exactly 
once. It follows from (A2) that there is always exactly one turnover equilibrium. In Fig. [5|\, this is illustrated for 
the matching pennies game with the same parameters as Fig. [2j3. 

If a/3 > 0, the line /3x* + 612 — 622 and the function g(x*) are either both increasing or both decreasing. Therefore, 
they can in principle intersect any odd number of times. Hence, it is possible to have multiple turnover equilibria, 
and these can appear or annihilate in pairs through cither saddle point bifurcations or pitchfork bifurcations. Fig. 
[SJ3-C shows a saddle node bifurcation in the coordination game with the same parameters as Fig. [SjD-E, except we 
have set \x equal to 0.8 and 1, respectively, to show a clearer bifurcation. 



Appendix B: Additional analysis of the lowest unique bid auction data 

In this Appendix, we present additional results on the lowest unique bid auction dataset used in Section [V] Fig. [6] 
shows how letting \ as an independent fitting parameter for each auction size leads to very good fits of the bidding 
distributions in all cases. The fitted values might suggest a higher turnover rate for larger auction sizes, which is 
reasonable considering for example that larger auctions typically involve larger bidding fees. Notice however that the 
relation is not strictly monotonic (fitted x for N — 135 is slightly larger than for N = 198). Moreover, it should be 
stressed that the prior distribution Xq has been obtained from the larger auction sizes, so that estimates of x m the 
large auction cases can be affected by overfitting. 

Finally, in Fig. [7] we show the evolution of the distance to the Nash equilibrium in series of auctions having similar 
number of players TV — 1398 ± 74. The exact date of each auction was not available, so that data are plotted simply 
as a function of the time ordering of the auctions. 

For each auction, we plot the normalized distance to the Nash equilibrium, defined as 

d = nY,{x7 ip - x? ash Y ( B1 ) 

i 

where x emp is obtained from the frequencies and -x Nash is the theoretical Nash equilibrium. It is easy to demonstrate 
that, if players bid according to the Nash equilibrium, the expected value of d is 1 (see [17]). A value of d significantly 
larger than one signals a departure from the Nash equilibrium. Fitting the data with a linear regression yields a slope 
m = 0.0008 ± 0.1001 that clearly does not allow for refuting the null hypothesis of no temporal trend. This suggests 
the presence of a mechanism keeping the population at a finite distance from the Nash equilibrium, as the turnover 
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FIG. 7: (Color online) Distance from the Nash equilibrium for 14 auctions having N = 1398 ±74, as a function of the time order 
of the auctions. The straight line is a fitted linear regression, with slope m = 0.0008 ± 0.1001 and intercept q = 7.27 ± 0.85. 



mechanism proposed in this paper. 
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